Skip to content

concurrent execution of evals - #170

Open
Andrew Kent (realark) wants to merge 2 commits into
mainfrom
ark/async-evals
Open

concurrent execution of evals#170
Andrew Kent (realark) wants to merge 2 commits into
mainfrom
ark/async-evals

Conversation

@realark

@realark Andrew Kent (realark) commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator
  • execute standard and remote evals concurrently
  • add options for otel export span limits (ran into span drop issues when evals got faster than span upload)
  • use virtual threads for java 21+ (and add jvm compat test for java 21)

Ran a few tests and performance scales as expected. 10 threads finish 10x faster

@realark Andrew Kent (realark) added the enhancement New feature or request label Aug 26, 2026
@realark
Andrew Kent (realark) force-pushed the ark/async-evals branch 15 times, most recently from 7c6f8b1 to 7d0e965 Compare September 2, 2026 05:05
@realark
Andrew Kent (realark) marked this pull request as ready for review September 2, 2026 05:28
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-02T05:33:16.745035Z 4813c31 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4813c31e6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Comment thread braintrust-sdk/src/main/java/dev/braintrust/eval/Eval.java Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant